Supervised learning in the presence of concept drift: a modelling framework

Abstract We present a modelling framework for the investigation of supervised learning in non-stationary environments. Specifically, we model two example types systems: prototype-based vector quantization (LVQ) classification and shallow, layered neural networks regression tasks. investigate so-called student–teacher scenarios which systems are trained from stream high-dimensional, labeled data. Properties target task considered to be due drift processes while training is performed. Different concept studied, affect density inputs only, rule itself, or both. By applying methods statistical physics, develop mathematical analysis dynamics Our results show that standard LVQ algorithms already suitable environments certain extent. However, application weight decay as an explicit mechanism forgetting does not improve performance under processes. Furthermore, gradient-based with sigmoidal activation functions compare use rectified linear units. findings sensitivity effectiveness differs significantly between function.